{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "2e4812da",
      "metadata": {},
      "source": [
        "# NonlinearOptimizer\n",
        "\n",
        "## Overview\n",
        "\n",
        "The `NonlinearOptimizer` class in GTSAM is a the base class for (batch) nonlinear optimization solvers. It provides the basic API for optimizing nonlinear factor graphs, commonly used in robotics and computer vision applications.\n",
        "\n",
        "The primary purpose of the `NonlinearOptimizer` is to iteratively refine an initial estimate of a solution to minimize a nonlinear cost function. Specific optimization algorithms like Gauss-Newton, Levenberg-Marquardt, and Dogleg and implemented in derived class.\n",
        "\n",
        "## Mathematical Foundation\n",
        "\n",
        "The optimization process in `NonlinearOptimizer` is based on iterative methods that solve for the minimum of a nonlinear cost function. The general approach involves linearizing the nonlinear problem at the current estimate and solving the resulting linear system to update the estimate. This process is repeated until convergence criteria are met.\n",
        "\n",
        "The optimization problem can be formally defined as:\n",
        "\n",
        "$$\n",
        "\\min_{x} \\sum_{i} \\| \\phi_i(x) \\|^2\n",
        "$$\n",
        "\n",
        "where $x$ is the vector of variables to be optimized, and $\\phi_i(x)$ are the residuals of the factors in the graph.\n",
        "\n",
        "## Key Methods\n",
        "\n",
        "- The `optimize()` method is the core function of the `NonlinearOptimizer` class. It performs the optimization process, iteratively updating the estimate to converge to a local minimum of the cost function.\n",
        "- The `error()` method computes the total error of the current estimate. This is typically the sum of squared errors for all factors in the graph. Mathematically, the error can be expressed as:\n",
        "    $$\n",
        "    E(x) = \\sum_{i} \\| \\phi_i(x) \\|^2\n",
        "    $$\n",
        "    where $\\phi_i(x)$ represents the residual error of the $i$-th factor.\n",
        "- The `values()` method returns the current set of variable estimates. These estimates are updated during the optimization process.\n",
        "- The `iterations()` method provides the number of iterations performed during the optimization process. This can be useful for analyzing the convergence behavior of the optimizer.\n",
        "- The `params()` method returns the parameters used by the optimizer. These parameters can include settings like convergence thresholds, maximum iterations, and other algorithm-specific options.\n",
        "\n",
        "## Usage\n",
        "\n",
        "The `NonlinearOptimizer` class is typically not used directly. Instead, one of its derived classes, such as `GaussNewtonOptimizer`, `LevenbergMarquardtOptimizer`, or `DoglegOptimizer`, is used to perform specific types of optimization. These derived classes implement the `optimize()` method according to their respective algorithms.\n",
        "\n",
        "## Parameters and Configuration \n",
        "\n",
        "The optimizer can be configured and instrumented using `NonlinearOptimizerParams`. Classes deriving from `NonlinearOptimizer` may choose to define their own parameters, inheriting from `NonlinearOptimizerParams`, specific to their optimizer implementation.\n",
        "Configurable parameters include\n",
        "- `maxIterations` -- The maximum iterations before optimizer termination (default 100)\n",
        "- `relativeErrorTol` -- The maximum relative error decrease to stop iterating (default 1e-5)\n",
        "- `absoluteErrorTol` -- The maximum absolute error decrease to stop iterating (default 1e-5)\n",
        "- `errorTol` -- The maximum total error to stop iterating (default 0.0)\n",
        "- `verbosity` -- The printing verbosity during optimization (default SILENT)\n",
        "- `orderingType` -- The method of ordering use during variable elimination (default COLAMD)\n",
        "- `linearSolverType` -- The type of linear solver to use in the nonlinear optimizer (default MULTIFRONTAL_CHOLESKY)\n",
        "- `iterativeParams` --The container for iterativeOptimization parameters. used in CG Solvers.\n",
        "\n",
        "### IterationHook\n",
        "The `NonlinearOptimizerParams` also contain an optional `iterationHook` field which, if set, is called by the optimizer after every iteration and allows custom behavior in response to the most recently completed iteration.\n",
        "The parameters provided in the iterationHook are\n",
        " - `size_t iteration` -- the iteration count\n",
        " - `double errorBefore` -- the error before the iteration started\n",
        " - `double errorAfter` -- the error after the iteration completed\n",
        "## Files\n",
        "\n",
        "- [NonlinearOptimizer.h](https://github.com/borglab/gtsam/blob/develop/gtsam/nonlinear/NonlinearOptimizer.h)\n",
        "- [NonlinearOptimizer.cpp](https://github.com/borglab/gtsam/blob/develop/gtsam/nonlinear/NonlinearOptimizer.cpp)\n",
        "- [NonlinearOptimizerParams.h](https://github.com/borglab/gtsam/blob/develop/gtsam/nonlinear/NonlinearOptimizerParams.h)\n",
        "- [NonlinearOptimizerParams.cpp](https://github.com/borglab/gtsam/blob/develop/gtsam/nonlinear/NonlinearOptimizerParams.cpp)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "colab_button",
      "metadata": {},
      "source": [
        "<a href=\"https://colab.research.google.com/github/borglab/gtsam/blob/develop/gtsam/nonlinear/doc/NonlinearOptimizer.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "license_cell",
      "metadata": {
        "tags": [
          "remove-cell"
        ]
      },
      "source": [
        "GTSAM Copyright 2010-2022, Georgia Tech Research Corporation,\nAtlanta, Georgia 30332-0415\nAll Rights Reserved\n\nAuthors: Frank Dellaert, et al. (see THANKS for the full author list)\n\nSee LICENSE for the license information"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "colab_import",
      "metadata": {
        "tags": [
          "remove-cell"
        ]
      },
      "outputs": [],
      "source": [
        "try:\n    import google.colab\n    %pip install --quiet gtsam-develop\nexcept ImportError:\n    pass"
      ]
    }
  ],
  "metadata": {
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}